Feeling for Sound: Mapping Sonic Data to Haptic Perceptions

This paper presents a system for exploring different dimensions of a sound through the use of haptic feedback. The Novint Falcon force feedback interface is used to scan through soundfiles as a subject moves their hand horizontally from left to right, and to relay information about volume, frequency content, envelopes, or potentially any analysable parameter back to the subject through forces acting on their hand. General practicalities of mapping sonic elements to physical forces are considered such as the problem of representing detailed data through vague physical sensation, approaches to applying forces to the hand that do not interfering with the smooth operation of the device, and the relative merits of discreet and continuous mappings. Three approaches to generating the force vector are discussed: 1) the use of simulated detents to identify areas of an audio parameter over a certain threshold, 2) applying friction proportional to the level of the audio parameter along the axis of movement, and 3) creating forces perpendicular to the subject’s hand movements. The potential uses of such a device are also discussed, such as ‘pre-feeling’ as a method for selecting material to play during a live performance, an aid for visually impaired audio engineers, and as a general augmentation of standard audio editing